The Power of Selective Memory: Self-Bounded Learning of Prediction Suffix Trees
نویسندگان
چکیده
Prediction suffix trees (PST) provide a popular and effective tool for tasks such as compression, classification, and language modeling. In this paper we take a decision theoretic view of PSTs for the task of sequence prediction. Generalizing the notion of margin to PSTs, we present an online PST learning algorithm and derive a loss bound for it. The depth of the PST generated by this algorithm scales linearly with the length of the input. We then describe a self-bounded enhancement of our learning algorithm which automatically grows a bounded-depth PST. We also prove an analogous mistake-bound for the self-bounded algorithm. The result is an efficient algorithm that neither relies on a-priori assumptions on the shape or maximal depth of the target PST nor does it require any parameters. To our knowledge, this is the first provably-correct PST learning algorithm which generates a bounded-depth PST while being competitive with any fixed PST determined in hindsight.
منابع مشابه
Self-Bounded Prediction Suffix Tree via Approximate String Matching
Prediction suffix trees (PST) provide an effective tool for sequence modelling and prediction. Current prediction techniques for PSTs rely on exact matching between the suffix of the current sequence and the previously observed sequence. We present a provably correct algorithm for learning a PST with approximate suffix matching by relaxing the exact matching condition. We then present a self-bo...
متن کاملSkip Context Tree Switching
Context Tree Weighting is a powerful probabilistic sequence prediction technique that efficiently performs Bayesian model averaging over the class of all prediction suffix trees of bounded depth. In this paper we show how to generalize this technique to the class of K-skip prediction suffix trees. Contrary to regular prediction suffix trees,K-skip prediction suffix trees are permitted to ignore...
متن کاملLearning to Use Selective Attention and Short-Term Memory in Sequential Tasks
This paper presents U-Tree, a reinforcement learning algorithm that uses selective attention and shortterm memory to simultaneously address the intertwined problems of large perceptual state spaces and hidden state. By combining the advantages of work in instance-based (or “memory-based”) learning and work with robust statistical tests for separating noise from task structure, the method learns...
متن کاملInstance-Based Utile Distinctions for Reinforcement Learning with Hidden State
We present Utile Suffix Memory, a reinforcement learning algorithm that uses short-term memory to overcome the state aliasing that results from hidden state. By combining the advantages of previous work in instance-based (or “memorybased”) learning and previous work with statistical tests for separating noise from task structure, the method learns quickly, creates only as much memory as needed ...
متن کاملLearning Visual Routines with Reinforcement Learning
Reinforcement learning is an ideal framework to learn visual routines since the routines are made up of sequences of actions. However, such algorithms must be able to handle the hidden state (perceptual aliasing) that results from visual routine’s purposefully narrowed attention. The U-Tree algorithm successfully learns visual routines for a complex driving task in which the agent makes eye mov...
متن کامل